In the past decade, digital imaging capabilities have been integrated into a wide range of devices, including digital cameras and mobile phones. Recently, the ability to capture stereoscopic images with these devices has become technically possible. Device manufacturers have responded by introducing devices integrating digital image processing to support this capability, utilizing single or multiple digital imaging sensors. A wide range of electronic devices, including mobile wireless communication devices, personal digital assistants (PDAs), personal music systems, digital cameras, digital recording devices, video conferencing systems, and the like, make use of stereoscopic imaging capabilities to provide a variety of capabilities and features to their users. These include stereoscopic (3D) imaging applications such as 3D photos and videos or movies.
To achieve stereoscopic image pairs that are precisely aligned, devices with a plurality of imaging sensors are often calibrated during the manufacturing process. The device may be placed into a special “calibration mode” on the manufacturing line, with the imaging sensors pointed at a target image designed to assist in clearly identifying each camera's relative position. Each camera of the device may then be focused on the target image and an image captured. Each captured image can then be analyzed to extract the camera's relative orientation.
Some cameras may be designed such that small adjustments to each camera's relative position can be made on the factory floor to better align the positions of the two cameras. For example, each camera may be mounted within an adjustable platform that provides the ability to make small adjustments to its position. Alternatively, the images captured by each camera may be analyzed by image processing software to determine the relative position of each camera to the other. This relative position data is then stored in a non volatile memory on the camera. When the product is later purchased and used, on board image processing utilizes the relative position information to electronically adjust the images captured by each camera to produce high quality stereoscopic images.
These calibration processes have several disadvantages. First, a precise manufacturing calibration consumes time during the manufacturing process, increasing the cost of the device. Second, any calibration data produced during manufacturing is static in nature. As such, it cannot account for changes in camera position as the device is used during its life. For example, the calibration of the multiple lenses may be very precise when the camera is sold, but the camera may be dropped soon after purchase. The shock of the fall may cause the cameras to go out of calibration. Despite this, the user will likely expect the camera to survive the fall and continue to produce high quality stereoscopic images.
Furthermore, expansion and contraction of camera parts with temperature variation may introduce slight changes in the relative position of each camera. Factory calibrations are typically taken at room temperature, with no compensation for variations in lens position with temperature. Therefore, if stereoscopic imaging features are utilized on a particularly cold or hot day, the quality of the stereoscopic image pairs produced by the camera may be affected.
Therefore, a static, factory calibration of a multi camera device has a limited ability to compensate for misalignments. While a periodic calibration would alleviate some of these issues, it may not be realistic to expect a user to perform periodic stereoscopic camera calibration of their camera during its lifetime. Many users have neither the desire nor often the technical skill to successfully complete a calibration procedure.
Furthermore, some devices enable the ability to capture a stereoscopic imaging pair using only one imaging sensor. With these devices, the device may be manually positioned, sometimes with the assistance of automated prompting, to capture the two images. Using this technique, misalignments between images captured as part of the stereoscopic image pair are very common. In these devices, there is no ability for a factory calibration to eliminate misalignments between the stereoscopic image pairs.
Because of persistent misalignment between the images of a stereoscopic image pair, methods have been introduced to provide high quality stereoscopic images from digital image pairs with some moderate misalignment. These methods digitally process the stereoscopic image pairs to produce aligned images. Aligning stereoscopic images may include cropping one or both images to correct for horizontal or vertical shift between the images of a stereoscopic image pair. The two images of a stereoscopic image pair may also be misaligned about a “z” axis, caused when one imaging sensor is slightly closer to a scene being imaged than the other imaging sensor. Cropping may also be required to correct for misalignment due to rotation of the images about a x, y, or z axis. Finally, cropping may also be required to adjust the convergence point of the two images in the stereoscopic image pair. These multiple crop operations may adversely affect the viewable region of the stereoscopic image. The resulting size of the viewable region of the resulting stereoscopic image pair may limit a device's ability to compensate for misaligned stereoscopic image pairs in some imaging environments